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In this paper we study the strengths and limitations of collaborative teams of simple agents. In 
particular, we discuss the efficient use of "ant robots" for covering a connected region on the Z 2 
■ grid, whose area is unknown in advance and which expands stochastically. Specifically, we discuss 

the problem where an initial connected region of So boundary tiles expand outward with probability 
p at every time step. On this grid region a group of k limited and simple agents operate, in order 
to clean the unmapped and dynamically expanding region. A preliminary version of this problem 
was discussed in [l], 0] , involving a deterministic expansion of a region in the grid. In this work we 
extend the model and examine cases where the spread of the region is done stochastically, where 
each tile has some probability p to expand, at every time step. For this extended model we obtain 
an analytic probabilistic lower bounds for the minimal number of agents and minimal time required 
to enable a collaborative coverage of the expanding region, regardless of the algorithm used and the 
robots hardware and software specifications. In addition, we present an impossibility result, for a 
variety of regions that would be impossible to completely clean, regardless of the algorithm used. 
Finally, we validate the analytic bounds using extensive empirical computer simulation results. 
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I. INTRODUCTION 



In this work we discuss in this work is the Cooperative Cleaners problem — a problem assuming a regular grid 
of connected rooms (pixels), some of which are 'dirty' and the 'dirty' pixels forming a connected region of the grid. 
J> ' On this dirty grid region several agents operate, each having the ability to 'clean' the place (the 'room', 'tile', 'pixel' 
or 'square') it is located in. We examine problems in which the agents work in stochastic dynamic environments — 
where probabilistic changes in the environment may take place and are independent of, and certainly not caused 
by, the agents' activity. In the spirit of Q we consider simple robots with only a bounded amount of memory 
(i.e. finite- state-machines) . 

The static variant of this problem was introduced in Q , where a cleaning protocol ensuring that a decentralized 
■ group of agents will jointly clean any given (and a-priori unknown) dirty region. The protocol's performance, in terms 
I \ of cleaning time was fully analyzed and also demonstrated experimentally. 

A dynamic generalization of the problem was later presented in in which a deterministic expansion of the 
environment is assumed, simulating the spreading of a contamination (or a spreading "danger" zone or fire). Once 
again, the goal of the agents is to clean the spreading contamination in as efficiently as possible. 

In this work we modify the 'dirty' region expansion model and add stochastic features to the spreadings of the 
region's cells. We formally define and analyze the Cooperative Cleaning problem, this time under this stochastic 
generalization. 

We focus on a variant of the Cooperative Cleaning problem, where the tiles have some probability to be contaminated 
by their neighbors' contamination. This version of the problem has applications in the "real" world and in the computer 
network environments as well. For instance, one of the applications can be a distributed anti-virus software trying to 
overcome an epidemic malicious software attacking a network of computers. In this case, each infected computer has 
some probability to infect computers connected to it. 

A more general paradigm of the cleaning problem is when the transformation of the contaminated area from one 
state to another is described in the form of some pre-defined function. For instance, following the previous example, 
we can say that the sub-network affected by the virus, is spreading by a certain rule. We can say that a computer 
will be infected by a virus with a certain probability, which depends on the number of the neighboring computers 
already infected. By defining rules for the contamination's spreading and cleaning, we can think of to this problem 
as a kind of Conway's "Game of Life", where each cell in the game's grid spreads its "seed" to neighboring cells (or 
alternatively, "dies") according to some basic rules. 

While the problem posed in [2J, as well as the analysis methods used and the correctness proofs, were all deter- 
ministic, it is interesting to examine the stochastic variant of such algorithms. In this work we analyze and derive a 
lower bound on the expected cleaning time for k agents running a cleaning protocol under a model, where every tile 
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in the neighborhood of the affected region may become contaminated at every time step with some probability, the 
contamination coming from its dirty neighbors. 

II. ORGANIZATION 

The main contributions of this paper are a bound on the contaminated region's size and a bound on the cleaning 
time which is presented in Section [Vj We also present a method which bounds the cleaning time for a given desired 
probability. We then provide an impossibility result for the problem raised in Section lVH The rest of paper is organized 
as followed: In Section IlIII we survey some of the related works. In Section [IVI we formalize the problem, giving the 
basic definitions needed for the later analysis, in Section [VIII we present some of the experimental results and compare 
them to the analytic bounds and concluding in Section [Villi 

III. RELATED WORK 

Significant research effort is invested in the design and simulation of multi-agent robotics and intelligent swarm 
systems (see e.g. f5l-flfjl|). 

In general, most of the techniques used for the distributed coverage of some region are based on some sort of 
cellular decomposition. For example, in the area to be covered is divided between the agents based on their 
relative locations. In [l2j a different decomposition method is being used, which is analytically shown to guarantee a 
complete coverage of the area. (13( discusses two methods for cooperative coverage (one probabilistic and the other 
based on an exact cellular decomposition). 

While some existing works concerning distributed (and decentralized) coverage present analytic proofs for the ability 
of the system to complete the task (for example, in |l^ - tl4| ). most of them lack analytic bounds for the coverage time 
(and often extensive amounts of empirical results on this are made available by extensive simulations). Although a 
proof for the coverage completion is an essential element in the design of a multi-agent system, analytic indicators for 
its efficiency are in our opinion of great importance. We provide such results, as bounds for the cleaning time of the 
agents, in Section IVI 

An interesting work to mention in this context is that of Koenig and his collaborators (l5l [l6j , where a swarm of 
ant-like robots is used for repeatedly covering an unknown area, using a real time search method called node counting. 
By using this method, the robots are shown to be able to efficiently perform a coverage mission, and analytic bounds 
for the coverage time are discussed. 

Another work discussing a decentralized coverage of terrains is presented in 17]. This work examines domains with 
non-uniform traversability. Completion times are given for the proposed algorithm, which is a generalization of the 
forest search algorithm. In this work, though, the region to be searched is assumed to be known in advance - a crucial 
assumption for the search algorithm, which relies on a cell-decomposition procedure. 

Vertex- Ant- Walk, a variation of the node counting algorithm is presented in Q and is shown to achieve a coverage 
time of 0(n(5c), where 8q is the graph's diameter, which is based on a previous work in which a cover time of 
0(?t, 2 (5g) was demonstrated fl8j . Another work called Exploration as Graph Construction, provides a coverage of 
degree bounded graphs in 0(n 2 ) time, is described in [19J. Here a group of ant robots with a limited capability 
explores an unknown graph using special "markers" . 

Similar works concerning multi agents systems may be found in [TTI4T3 . [2p| - [25j ). 

The Cooperative Cleaning problem is also strongly related to the problem of distributed search after mobile and 
evading target ( s) [26l429j j or the problems discussed under the names of "Cops and Robbers" or "Lions and Men" 
pursuits |30l - l35l |. 

IV. DEFINITIONS 

In our work we will use some of the basic notations and definitions, that were used in 0], which we shell briefly 
review. As in the above mentioned previous works on this problem, we shall assume that the time is discrete. 

Definition 1. Let an undirected graph G(V,E) describe the two dimensional integer grid Z 2 , whose vertices (or 
"tiles ") have a binary property called "contamination". Let cont t (v) denote the contamination state of the tile v at 
time t, taking either the value "on" (for "dirty" or "contaminated" ) or "off" (for "clean"). 
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For two vertices v, u G V , the edge (v, u) may belong to E at time t only if both of the following hold : a) v and u 
are A— Neighbors in G. b) cont t (v) = cont t (u) = on. This however is a necessary but not a sufficient condition as we 
elaborate below. 

The edges of E represent the connectivity of the contaminated region. At t = all the contaminated tiles are 
connected, namely : 

(v,u) € Eq (v,u are A — Neighbors in G) A (conto(v) = conto(u) = on) 

Edges may be added to E only as a result of a contamination spread and can be removed only while contaminated 
tiles are cleaned by the agents. 

Definition 2. Let F t (V Ft ,E t ) be the contaminated sub-graph of G at time t, i.e. : 

Vp t = {v € G | cont t (v) = on} 

We assume that F is a single simply-connected component (the actions of the agents will be so designed that this 
property will be preserved). 

Definition 3. Let OF denote the boundary of F . A tile is on the boundary if and only if at least one of its ^-Neighbors 
is not in F, meaning : 

dF = {v | v e F A 8- Neighbor s(v) n (G \ F) ^ 0} 

Definition 4. Let S t denote the size of the dirty region F at time t, namely the number of grid points ( or tiles ) in 
F t . 

Let a group of k agents that can move on the grid G (moving from a tile to its neighbor in one time step) be placed 
at time to on Fo , at some point po € Vp t ■ 

Definition 5. Let us denote by A Ft the potential boundary, which is the maximal number of tiles which might be 
added to F t by spreading all the tiles of dF t . 

AF t = {v : 3u e dF t and v e 4 — Neighbor s(u) and v ^ F t } 

As we are interested in the stochastic generalization of the dynamic cooperative cleaners model, we will assume 
that each tile in AFt might be contaminated with some probability p. In the model we will analyze later, we assume 
that the status variables of the tiles of AF t are independent from one another, and between time steps. 

Definition 6. Let us denote by $„ (v) the surrounding neighborhood of a tile v, as the set of all the reachable tiles 
u from v within n steps on the grid (namely, the "digital sphere" or radius n around v). In this work we assume 4~ 
connectivity among the region cells — namely, two tiles are considered as neighbors within one step iff the Manhattan 
distance between them is exactly 1. 

The spreading policy, S (v, <p, t), controls the contamination status of v at time t + 1, as a function of the contami- 
nation status of its neighbors in its n-th digital sphere, at time t. Notice the S (v, <fi, t) can be also non deterministic. 

Definition 7. Let us denote by S (v, <j>, t) the spreading policy of v as follows: 

S (v, 4,, t) : (V, {On, Off} a , N) -+ {On, Off} 

Where a = \$> n (v)\, V denotes the vertices of the grid, {On, Off} a denotes the contamination status of the members 
of $„ (v), at time t (for t E N). 

A basic example of using the previous definition of S () is the case of the deterministic model, where at every d 
time-steps the contamination spreads from all tiles in dF t to all the tiles in 5F t . This model can be defined using the 
S function, as follows: 

For every tile v we first define $„ (v) where n equals to 1 and assuming 4-connectivity. Then S (v, (f>, t) , for any 
time-step t will be defined as follows: 

„ , . _ J On lit mod d = and v G AF t 
s.(v,<f>,t)-^ QS 0therwise 

Notice that due to the fact that we are assuming that v is in AF t , its surrounding neighborhood contains at least one 
tile with contamination status of On. 
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An interesting particular case of the general S() function is the simple uniform probabilistic spread. In this scenario, 
a tile in V € AF t becomes contaminated with some predefined probability p, if and only if at least one of its n-th 
neighbors are contaminated at time step t. Using the S() function and the probability p, this can be formalized as 
follows : 

For every tile v we first define $„ (v) where n equals to 1 and assuming 4-connectivity. 

„ , , . J On with probability of p if v £ AF t 
" ^' ^ = \ Off Otherwise 

This model can naturally also be defined simply as : 

Vi € N, Vw € Af t , Profc (corai t +i(v) = On) = p 

In our work we will focus on this model, while deriving the analytic bounds for the cleaning time. 



V. LOWER BOUND 



A. Direct Bound 



In this section we form a lower bound on the cleaning time of any cleaning protocol preformed by k agents. We 
start by setting a bound on the contaminated region's size at each time step, St- As we are interested in minimizing 
the cleaning time we should also minimize the contaminated region's area. Therefore we are interested in the minimal 
size of it, which achieved when the region's shape is sphere [2|]. In our model each tile in the potential boundary, AF t , 
has the same probability p to be contaminated in the next time step. The whole probabilistic process in each time 
step is Binomial Distributed, under the assumption that the spreading of each tile at any time step is independent 
from the spreadings of other tiles or from the spreadings of the same tile at different time steps. 

As we are interested in the lower bound of the contaminated region's size we will assume that the expected number 
of newly added tiles to the contaminated region is minimal, which occurs when the region's shape forms a digital 
sphere (as presented in [36]). Then we can compute the expectation of this process for a specific time step t. Therefore, 
the size of the potential boundary is AF t = 2y/2 ■ St — I as shown in [36L Wl\ . 

Definition 8. Let us denote by X t the random variable of the actual number of added tiles to the contaminated region 
at time step t. 

Assuming the independence of tiles' contamination spreadings and given St, X t is Binomial Distributed, X t \St ~ 
B (AF t ,p), where each tile in the potential boundary has the same probability p to be contaminated. Therefore, we 
can say that the expectation of X t given S t is p = E (X t [S t ) = p ■ AF t . 

Notice that occasionally the number of new tiles added to the contaminated region may be below /x. As we are 
interested in a lower bound, we should take some p! < p such that: Pr[X t < p'\S t ] < e, meaning that the probability 
that the number of the newly added tiles to the contaminated region is smaller than p' is extremely small (tends to 
0). 

In order to bound Xt by some \j! we shall use the Chernoff Bound , where 6 is the desired distance from the 
expectation, as follows : 

Pr [X t < (I - S)p\S t ] <e~^ 

Definition 9. Let us denote by q t the probability that at time step t, the size of the added tiles to the contaminated 
region is not lower than p! = (1 — 8)fi and it can be written as follows : 

q t = (1 - Pr [X t < (I — 6)p\S t ]) 

Theorem 1. Using any cleaning protocol, the area of the contaminated region at time step t can be recursively lower 
bounded, as follows : 



Pr 



S t +i > S t - k- 



2 • (1 - S)p ■ ^2 • (St -k)-l\ \S 



Proof. Notice that a lower bound for the contaminated region's size can be obtained by assuming that the agents are 
working with maximal efficiency, meaning that each time step every agent cleans exactly one tile. 
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In each step, the agents clean another portion of k tiles, but the remaining contaminated tiles spread their contam- 
ination to their 4— Neighbors and cause new tiles to be contaminated. 

Lets us denote by the random variable St+i the number of contaminated tiles in the next time step. Using 
Definition |S] we can express St+i as follows : 

S t+ i — S t - k + X t 

Lets first bound the number of the added tiles using the Chernoff Bound . As Xt given St is Binomial Distributed, 
X t \S t ~ B (AF t ,p) and fi = E (X t \S t ) = p ■ AF t . Using Chernoff Bound we know that: 



Pr [X t < (1 - S)n\S t ] < e-V =>. Pr [X t < (1 - 5)p ■ AF t \S t ] < e -^^ 
Assigning X t — St+i ~ St + k from former definition of S*+i, we get: 

Pr [S t+ i -S t + k<(l-S)p- AF t \S t ] < e-^^ 

As we are interested in the minimal number of tiles which can become contaminated at this stage. The minimal 
number of 4 — Neighbors of any number of tiles is achieved when the tiles are organized in the shape of a "digital 
sphere" (see [III, |37]]) - i.e. the potential boundary is AF t — 2^/2 ■ (S t — k) — 1. Assigning AF t value: 



Pr 



S t+ i < S t - k + (1 - S)p ■ 2y/2- (S t -k)-l\S t 



< e 



5 2 ■p-2y / 2(S t -fc)-l 



As we are interested in the complementary event and using Definition [5] 



Pr 



St+i >S t -k + (l-S)p- 2^2 • (S t - k) - l\S t 



> 1 



■5 2 p-2 v /2(S t -fc)-l 



(i) 



As the number of tiles must be an integer value, we use I (1 — 5) ■ p ■ 2\J2 ■ S t — lj to be on the safe side. Using 
inequality [T] we get : 



Pr 



St+i > St — k + 



2-(l-8)p-^2-(S t -k)-\ \S t 



> 



Qi 



□ 



Notice that as illustrated in Figure l(a)| which demonstrates the bound presented in Theorem [lj as S decreases the 
produced bound for the stochastic model is closer to the bound of the deterministic model, for d = - . 

Definition 10. Let us denote by Qt the bound probability that the contaminated region's size at time step t will be 
[at least] St- Qt can be expressed as follows : 



Qt 



i=0 



Notice that for bounding the area of the region at time step t using Theorem Q] and Definition [TOl the bound, which 
will be achieved, will be in probability Q t . We want to have Q t sufficiently high. 

We shall assume, for the sake of analysis, that the dynamic value of the area, St, is always kept not less than some 

S < S - k + 2-(l-5)p- y/2 ■ (So - k) - lj (as we want S\ to be bigger or equal to S). Then the next Lemma 

holds : 

Lemma 1. For any T > 1, if for all 1 < t < T the contaminated region's size St is always kept not less than some 
S<S -k+ 2 ■ (1 -8)v J2 ■ (So - k) - 1 then : 



Qt > Qt = f = ( 1 
Proof. We will prove this Lemma by induction on T. 
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(a)A lower bound for the contaminated region St, the area at time t, 
for various values of <5 




FIG. 1: An illustration of the bound presented in Theorem [T] In (a) we can see the deterministic model (the Zig-Zag line) 
with spread each 3 time steps (d = 3) compare to the stochastic model with p = 1/3 and S £ [0.01, 0.1, 0.2, 0.3, 0.5] where both 
models have k — 150 and start with So = 20000. In (b) we have the lower bound for So = 20000, p = 0.5 and 8 — 0.3 for 
different number of agents k G [150, 160, 170, 180, 190]. 



Base step: For T = 1, we can write the probabilities Qi and Qi as follows : 

Qi = qi > 1 - e- 4a -"-V2-(5i-fc)-i 



Qi =q> i_ e - 4a W2-(S-fc)-i 
As we assume that St> S than it is not hard to see that Q\ >Q\. 
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• Induction hypothesis: We assume that for all T < T' holds Qt > Qt- 

• Induction step: We will prove that for T = T' + 1 holds Qt > Qt- From definition [10] we can write the 
probability Qt as follows : 

Qt = Qt'+i = Qt 1 ■ qr'+i 

By the induction hypothesis we know that for all T < T" holds Qt > Qt than we can rewrite Qt'+i and also 
writing Q T for T = T + 1 we get that : 

Qt = Qt'+i > Qt< ■ Qt'+i 



Qt = Qt'+i = Qt> ■ q 

As we want to compare these probabilities and to prove that Qt > Qt all we need to show is that qr'+i > $• 
As we assume that for all 1 < t < T' + 1 holds that S t > S and particularly for t = T' + 1, than it is not hard 
to see that Qt'+i > q and therefore Qt > Or- 

□ 

Theorem 2. i*br anj/ contaminated region of size So, using any cleaning protocol, the probability that S T ~ S , the 
contaminated area at time step t — f$, is greater or equal to some S < Sq — k + 2 • (1 — S)p- ^J2 ■ (So — k) — lj can 
be lower bounded, as follows : 



Pr 



Sf s > S 



> 1 - e 



-8 2 -p-2^2-{S-k)-l^ 



where 



vj ■ { S — k — i 



ru ■ (S*o — k — |) + In 



T5 



^•(so-fc-ij-i 



anrf 



tu^2(l-5) 2 -p 2 



Proof. Observe that by denoting j/t = St Theorem [T] can be written as 



2.(i- ( 5)-p v / 2-(y t -fc)-iJ 



2/t+i - y* > 

Searching for the minimal area we can look at the equation : 

Vt+i ~ Vt = 

By dividing both sides by At = 1 we obtain 



2-(l-S)-py/2-(y t -k)-l 



vt+i -yt = y' = 



(l-S) 2 -p 2 



(2) 



Notice that the values of y', the derivative of the change in the region's size, might be positive (stating an increase 
in the area), negative (stating a decrease in the area), or complex numbers (stating that the area is smaller than fc, 
and will therefore be cleaned before the next time step). 

Let us denote x 2 = (1 — <5) 2 • p 2 ■ 8 \y — (k + |)1 . After calculating the derivative of both sides of this expression 
we see that : 



2x-x'= (1 -5) 2 -p 2 ■ 8y' 
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and after using the definition of y' of Equation [2] we see that : 



2x-^=2x-x' = (l~S) 2 -p 2 



\l-5) 2 -p 2 



y- [k + 



< (l-<5) 2 -p 2 -8 {x-k) 
From Equation [3] a definition of dt can be extracted : 

1 2x 



(3) 



dt > 



> 



dx 



(1 — S) 2 ■ p 2 x — k 
1 x — k + k 



dx > 



1 



4(l-S) 2 -p 2 x-k 4(1 - S) 2 ■ p 2 \ x-k 



dx 



The value of x can be achieved by integrating the previous expression as follows (notice that we are interested in 
the equality of the two expressions) : 



dt -- 

£q J X 



1 



4(l-*)a.pa 



1 



k 



- k 



dx 



After the integration we can see that : 

t 



to 



4(1 -S) 2 -p 2 



(x + k In (x — k)) 



and after assigning to = : 



4(1 - 8f ■ p ■ t = x - x Q + k In 



x — k 
xq — k 



Returning back to y and using w definition we get 



w ■ t = \\w \ y — k — — ] — \ zu [yo — k — — ] + In 



™{y-k- |) - | 



^Jw(y -k-\) - | 



Returning to the original size variable St, we see that : 



w ■ t = \ lw \ St — k — — ] — * U7 [ Sn — k — — ] + In 



n fe 

2 



i\ fe 

2 , 



(4) 



Defining that ts — t and combining EQ. @] with Lemma [T] knowing that Sf > S for all 1 < t' < f$ we get the 
following inequality: 



Pr 



Sr s > S 



> 



(l - e -<5 2 -p-2V2-(S-fc)-l^ ' 



where : 



y^ 7 ■ (S - k - |) - ^ ■ (S* - fc - |) 



In 



^■(g-fc-i)-* 



and 



4 2(1 - <S) 2 • p 2 



□ 
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600 




(b)The bound on t for various number of agents k as a function of S 



FIG. 2: An illustration of the bound presented in Theorem [2] of the cleaning time t in order to reach S. In |(a)| we can see 
the bound on the cleaning time where p = 0.5 and 8 £ [0.27, 0.3, 0.4, 0.5, 0.6, 0.7] where the cleaning done by k = 150 agents 
and starting with So = 20000. In (b) we have the lower bound on the cleaning time for So = 20000, p = 0.5 and S = 0.3 for 



different number of agents k £ [150, 160, 170, 180, 190] 



In Theorem [2] we can guarantee with high probability of Q T5 that the contamination region's size will not be lower 
than S - namely for any time step t > t$ the probability Pr[St > S] is getting lower and therefore the probability 
that the agents will succeed in cleaning the contaminated area is increasing. We are showing that by choosing small 
enough S so we know that the agents will succeed in cleaning the rest of the 'dirty' region, we will be able to guarantee 
with high probability the whole cleaning of the 'dirty' region. For example, choosing S to be in o(k) will assure that 
for S t < S < c ■ k for some small constant c, the rest of the contaminated region will be cleaned in at most c time 
steps by the k cleaning agents. 

An illustration of the bound on the cleaning time, as presented in Theorem [2] is shown in Figure [2] and the 
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1 

S ^o 4 

(a)The bound probability Qt as a function of S for various 5 values 




(b)The bound probability Qt as a function of S for various p values 



FIG. 3: An illustration of the probability produced by the bound presented in Theorem [2] In (a) we can see the bound 
probability Q t for initial region's size So = 20000, spreading probability p — 0.5 and number of agents k — 150 for the following 
of values of 5 £ [0.4,0.3,0.25,0.2]. In [(b)] we can see Qt for So = 20000, 6 = 0.3 and k = 150 for the following of values of 
p G [0.6, 0.5, 0.4, 0.3] 



corresponding bound probability in Figure [3] Notice that as S and 5 increase the cleaning time decreases. 



B. Using The Bound 



In Theorem [2] we presented a bound which guarantees that the contaminated region's size will not be smaller than 
some predefined size S with the bound probability, Q t . We suggest a method which make this bound useful when 
one willing to be guaranteed of successfully cleaning of the contaminated area with some desired probability Q t in a 
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certain model, where the initial contaminated region's size is So, each one of the tiles in the surrounding neighborhood 
of the contaminated area has a probability p to be contaminated by each one of its neighbors's contamination spreads 
and with k cleaning agents. 

Notice that the only free variables left in the bound are the analysis parameters 8 and S. We should also notice 
the fact that as 8 decreases the "usefulness" of the bound decreases (see Figure [3]) because when 8 is closer to 0, the 
model tern to the deterministic variant of the cooperative cleaning problem. In this variant of the problem the bound, 
as presented in Theorem [21 will "predict" that the contamination will spread exactly by the potential boundary mean 
in every step (as shown in IV Al) . Furthermore, as S increases, although the "usefulness" of the bound increases, the 
predicted bound is the naive one, where at each step there are no spreads and all the k agents clean perfectly - i.e. 
the size of the contaminated region at time step t will be exactly St = So — t ■ k (which can be guaranteed in high 
probability) . 

We suggest the following method, in order to eliminate the need to identify the analysis parameters. Once someone 
willing to use this bound he should provide the desired bound probability - Q t and the parameter of the model. Then 
for each value of 8 in the range of [0, 1] he should find the corresponding value of the minimal S which satisfies the 

inequality Pr St > S > Qt as illustrated in Figure [4] As 8 € R - i.e. a real number, once using this method we 

should choose the granularity of i5 for which calculate the appropriate S. 

Notice that there can be exists some minimal value of 8 = Smin, where for any value of 8 < Smin there is no 
solution for the bound inequality. Also notice that there can be exists some maximal value of 8 = 8 m ax, where for 
any value of 8 > Smin the corresponding S is the same as for 8 max- 

Furthermore, for each pair of values of 8 and S there exists its corresponding cleaning precess of k agents with initial 
region's size So as demonstrated in Figure [4(b)] Each curve bounds the cleaning process from So to the applicable S. 

As we are interested in finding the tightest bound, looking at the frontier of the bounds, as shown in Figure [4(b)| we 
can combine the relevant curves to one comprehensive bound. This bounds integrates the bound for a specific range 
of j values of 8 G [8i i: 8i 2 , 5^] , where for each time step t we choose the maximal St as illustrated in Figure [B( a) | 

Notice that the combined bound is independent of the selection of values for analysis parameters 8 and s. Also we 
can notice that this bound limits the contaminated region's size to some minimal Smi n where we almost certain that 
the agents will succeed in terminating the cleaning process successfully. 

Notice that the inequality in Theorem [5] bounds the probability that the contaminated region's size at time step t 
will not be smaller than S , therefore, an increase in the spreadings probability causes to an increase in the expected 
contaminated region's size and thus increases the probability (qf (as demonstrated in Figure [3(b) 1 



Notice that as illustrated in Figure [5 (b)[ as the number of agents increases the probabilistic and the deterministic 
bounds are more similar. This result is not surprising considering the method we presented. In our method, in order 
to make the bound tighter, we favor the lower values of 8. As the number of agents increases the bound can be 
guaranteed in the desired probability with lower values of 8 and as 8 decreases the model becomes more similar to 
the deterministic one. 



C. Parameters Selection 



One of the problems of the bound as brought in Section IV Al is the nature of the probabilistic bounds to decay 
to (as shown in Figure [3j), which caused due to the fact that the bound probability Q t is a product of each step's 
probability, q t , and because as t increase Q t decreases. One of the reasons which explains this problem is a bad 
selection of parameters - e.g. in the bound for the cleaning time (Eq. [4] a selection of too small S will lead to fast 
decay in the probability. Furthermore, there exist trade-offs, when selecting the parameters' values, between the 
bound results and the the probability which guarantees its likelihood (e.g. see Figure [7]). 

One way to avoid this problem is by selecting S as big as possible, as in Figures [SJind [7] As S increases the 
probability of each time step, q t , increases and so the total probability Q t . Another technique for eliminating this 
problem is by "wrapping" number of time steps into one, thus artificially decreasing the time and therefore decreasing 
the power of qt in Q t (in Lemma [I]). 

Another example of the trade-off in choosing the parameters can be shown in Figure |3(b)| where Qt is illustrated 
for various values of the probability p. Interestingly, as p decreases our confidence in the bound result is decreasing 
although we know that the the agents preforming the cleaning protocol have a better chance to successfully complete 
their work. 
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Q, = 0.95 
- Q. = 0.90 



- Q ( = 0.85 




(a)S as a function of 5 for various bound probability Qt values 



x 10 




(b)The region's size St for various values of < Qt, S > pairs 



FIG. 4: Figure [(a)] is an illustration of S as a function of S using the bound prese nted in Theorem [2] for the following model 
parameters - So — 20000, p = 0.1 and k = 50 for various values of Qt- Figure (b) is an illustration of the cleaning process for 



various pairs of values of S and S as shown in Figure (a) for the same model parameters with Qt = 0.95 



VI. IMPOSSIBILITY RESULT 



While the theoretical lower bound presented in Section [V] can decrease the uncertainty whether a solution for the 
cleaning problem with certain number of agents exists, one might be interested in the opposite question, namely — how 
can we guarantee that a group of agents will not be able to successfully accomplish the cleaning work (regardless of 
the cleaning protocol being used or the contaminated region's properties e.g. its shape and spreading probability). A 
first impossibility result for the deterministic case of the Cooperative Cleaners problem was published in [Hj]), where 
an initial size that is impossible to clean (using any protocol) was demonstrated. In this paper, we extend this result 
in order to be applicable for stochastically expending domains as well. Consequently, wc will set the impossibility 
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(a)The combined bounds for various values of the bound probability 

Qt 




k 

(b)The ratio between the bounds as function of the number of agents k 



FIG. 5 

parameters 



Figure (a) An illustration of the combined bound for the bounds as shown in Figures 4(a) and 4(b) 
20000, 



So 



0.1 and k = 50 for various values of Q t 



for the same model 
compared to the deterministic model with d = 10 and 



to the naive bound So/k. Figure (b) compare the deterministic bound and the probabilistic bound as a function of the desired 
guaranteeing probability - Q t for various contaminated region's sizes and number of cleaning agents. 



result with probabilistic restrictions as follows : 

Theorem 3. Using any cleaning protocol, k agents cleaning a contaminated region , where each tile in the potential 
boundary may be contaminate by already contaminated neighboring tiles with some probability p in each time step, 
will not be able to cleat this contaminated region if : 



So > 



S-p 2 
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(a)The ratio between the bounds as function of the bound probability 
Qt for various number of agents k 
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(b)The ratio between the bounds as function of the bound probability 
Qt for various values of initial region's size So 



FIG. 6: Figures (b) and (a) compare the deterministic bound and the probabilistic bound as a function of the desired 



guaranteeing probability - Q t for various contaminated region's sizes and number of cleaning agents. 



with the probability Q t for any time step t - i.e. : 

VtPr [S t > S ] = Q t 

Proof. Firstly, we shell require that the contaminated region's size increases between each time step, guaranteeing us 
that the contaminated region's size will keep on growing, and thus impossible to be cleaned. Therefore we want that 
in each time step t the size of the contaminated region will be bigger than the previous one - i.e. : 



S t +i -S t >0 
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(a)The bound probability Qt as a function of S for various p values 
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(b)The bound probability Qt as a function of 5 for various numbers of 
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FI G. 7 : Parameters selection. In (a) we can see Qt for So = 20000 and k — 150 for the following of values of p £ [0.4, 0.3, 0.2, 0.1] . 
In [(b)] we can see Q t for So = 20000 and p = 0.4 for the following of values of k 6 [150, 250, 350, 450, 550, 650] 



Using Theorem [T] we know that : 

St+i > S t - k 
and therefore we shell require that : 



2-(l-6)p-y/2-(S t -k)-l 



2-(l-tf)j>- y/2 ■ (S t -k)-l -fc>0 
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Choosing, without loss of generality, t = and after some arithmetics, we see that : 



So > 



8(1-5) V 



As Sq as a function of S is monotonically increasing and tends to infinity as S tends to 1, we can lower bound Sq with 
5 = 0, therefore : 



So > 



1 



P 



(5) 



We would like this process to continue for t time steps, thus applying the same method for all time steps and using 
Def. [101 we get that : 



VtPr [S t > S ] = Q t 



□ 



□ 



Notice that Theorem [3] produces two results - the first one is the minimal initial region's size Sq which guarantees 
that the cleaning agents will not be able to successfully accomplish the cleaning process and second one is the 
corresponding probability Q t in which this Sq can be guaranteed. Also notice that in order to evaluate Q t , one should 
use the appropriate Sq and 5. 

Interestingly, the results demonstrated in Figure. [5] of the impossibility result as presented in Theorem [3J as the 
number of cleaning agents increases the probability which we can guarantee the minimal initial region's size Sq also 
increases - in other words, although as the number of agents increases, the corresponding minimal initial region's size 
also increases and the probability in which we can guarantee that the agents will not be able to successfully clean the 
region increases as well. 



VII. EXPERIMENTAL RESULTS 



In previous work [2| a cleaning protocol for a group of K agents collaboratively cleaning an expending region was 
developed, called SWEEP . The performance of the algorithm was analyzed in [2| , both analytically and experimen- 
tally. We had implemented the algorithm for a revised environment - where stochastic changes take place, as defined 
throughout this paper. We have conducted an extensive observational analysis of the performance of this algorithm. 
Exhaustive simulations were carried out, examining the cleaning activity of the protocol for various combinations of 
parameters - namely, number of agents, spreading probability (or spreading time in the deterministic model) and 
geometric features of the contaminated region. All the results were averaged over at least 1000 deferent runnings 
in order to get a statistical significance. In the deterministic model we average the results over the all the possible 
starting positions of the agents. Notice that, in order to minimize the running time, all running were stopped after 
some significant time and we consider these runnings as failure - i.e. these results are not included in the average 
calculation and not counted in the success percentage. 

Some of the experimental results are presented in Figure [5] comparing the probabilistic model and the deterministic 
model over three deferent shapes (circle, square and cross) with range of number of agents. Notice the interesting 
phenomenon, where adding more agents may cause to an increase in the cleaning time due to the delay caused by the 
agents synchronization in the SWEEP protocol. 



VIII. CONCLUSIONS 



In this work we set the foundations of the stochastic model for the Cooperative Cleaning problem and introduce, for 
the first time, the basic definitions describe this problem. We present two lower bounds on the contaminated region's 
size and on the cleaning time under the limitation of this probabilistic model and demonstrate an impossibility result 
on the number of agents which are essential for a successful completion cleaning a contaminated region. 

One of the ways these results could be further enhanced would involve analyzing the transition process between 
several possible "states" of the system, as a Markov process. Once analyzing the process as a Markov's Chain, we 
can get the stationary distribution of the process i.e. the probability to get to each one of the ending states (totally 
clean or impossible to clean). 
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x 10 




(a)The minimal initial region's size So as a function of the number of 
agents k for various probabilities. 




(b)The guaranteed probability Qt as a function of the number of 
agents k for various probabilities. 



FIG. 8: An illustration of the impossibility result as presented in Theorem [3] In (a) we can see the minimal initial region's 



size So for number of agents k = [1..60] and spreading probability p = [0.1,0.2,0.3,0.4]. In (b) we can see the corresponding 
probability Qt. 



It is also interesting to mention the similarity of this work to recent works done in the field of influence models in 
social networks. For example, in [38| the authors demonstrate that a probabilistic local rule can efficiently simulate 
the spread of ideas in a social network. Combining this result with our work can generate a unique approach for 
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(a)Average TsucCESS as a function of the number of agents k 




(b) Success percentage as a function of the number of agents k 



FIG. 9: Experimental results for various number of agents for spheric and squared contaminated region with starting size 
So = 500 and with spreading probability of p — 0.02 (notice that all the running were stopped after 3000 time steps). In (a) 
can see the results compared to the deterministic model results (with d = = and in (b) the success percentage 



analyzing dynamics of information flow in social networks. 
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